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Abstract. This paper is about spectral properties of transfer operators for 
contact Anosov flows. The main result gives the essential spectral radii of 
the transfer operators acting on an appropriate function space exactly and 
improves the previous result in [16]. Also we provide a simplified proof by 
using the so-called FBI (or Bargmann) transform. 



1. Introduction 

In this paper, we consider spectral properties of transfer operators for contact 
Anosov flows. A contact Anosov flow is by definition an Anosov flow preserving a 
contact form on the underlying manifold. The geodesic flows on closed negatively 
curved manifolds are typical examples of contact Anosov flows and have been stud- 
ied extensively since the work[7] of E. Hopf in 1930's. We refer [2, 14, 4, 9, 16] and 
the references therein for succeeding works related to this paper. 

For a contact Anosov flow F* : M — > M and a multiplicative cocycle g l over it, 
we consider the one-parameter family of transfer operators 

£*it = g l -uoF 1 . 

In the previous paper[16], we studied the case g* = X, that is, the case of pull-back 
operator and proved that the operators £' for sufficiently large t are quasi-compact 
if we choose an appropriate function space for them to act on. This implied not 
only exponential decay of correlations, which had been proved by Dolgopyat[4] 
and Liverani[9], but also a precise asymptotic formula for correlation decay or the 
Rucllc-Pollicott resonance. The main result of the present paper generalizes it to 
transfer operators with general cocycles g and improves the statement slightly by 
giving the essential spectral radius of £* exactly in terms of dynamical exponents. 
Besides we provide a simplified proof for the main result by introducing a technique, 
the FBI transform, from semi-classical analysis. 

The basic idea behind our argument is to regard functions on the manifold M 
as superpositions of wave packets (that is, localized simple wave functions) and 
to study the action of transfer operator £' on each of those wave packets. The 
wave packets are parametrized by elements of the cotangent bundle T*M, that is, 
pairs of a point x G M, which indicates the center of mass, and a cotangent vector 
£ £ T*M, which indicates the frequency vector. The action of the transfer operator 
£* of the wave packets are closely related to the action (DF 1 )* : T*M -> T*M of 
the flow F* on the cotangent bundle T*M . That is, roughly speaking, the wave 
packet corresponding to (x, £) € T*M is transferred by £* to a superposition of 
wave packets corresponding to elements of T*M close to (DF t )*(x, £). Notice that 
the action of the flow F on T*M is not recurrent outside any small neighborhood 
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of the one-dimensional subbundlc spanned by a and also that the part of £' acting 
on wave packets with low frequency should be compact. Therefore, to estimate 
the essential spectral radius of £*, we mainly concern the action of £* on wave 
packets that have high frequency in the direction of the contact form a. That is, in 
the case of geodesic flows, we concern the situation where wave packets with high 
frequency are proceeding along the geodesic curves in the corresponding directions. 
This reminiscent us of the situation studied in semiclassical analysis and suggests 
a vague idea that some argument and technique in semi-classical analysis may be 
useful in the study of "classical" geodesic flows (or contact Anosov flows, more 
generally). In this paper, we would like to show that this is the case in fact. See 
also the recent paper [6] for a similar approach, in which Faure and Sjostrand use 
semi-classical analysis to give an asymptotic estimate for the eigenvalues of the 
generator of transfer operators. 

2. The main result 

Let M be a closed odd-dimensional C°° Riemann manifold of dimension 2d + 1. 
We suppose that M is equipped with a C°° contact form a, which is by definition 
a differential 1-form such that a A (da) d vanishes nowhere on M. A C°° flow 
F* : M — > M is said to be Anosov if there exist constants C > 1, A > 1 and a 
continuous invariant decomposition TM = E C (BE S (BE U of the tangent bundle such 
that 

(1) E c is the 1-dim subbundle spanned by the generating vector field V of F*. 

(2) ||£>F*(V)|| < CAo*||u|| for any v e E s and t > 0. 

(3) ||£)F-*(t))|| < CA^IHI for any v e E u said t > 0. 

If an Anosov flow F* : M — > M preserves the contact form a, we call it a contact 
Anosov flow. For a contact Anosov flow, the subspaces E s and E u is contained 
in the kernel of a because of invariance of a and, hence, kera = E s © E u . The 
restriction of da to ker a is a symplectic form from the definition of contact form 
and vanishes on E u and E s because it is invariant with respect to the flow F* . This 
implies dimF s = dimF„ = d in particular. 

A C°° one-parameter family of functions g t : M — > C \ {0} with parameter 
t E K is called a multiplicative cocycle over a flow F* : M — > M if g t+s (x) = 
g t (F s (x)) ■ g s (x) for t, s £ R and x E M. For a flow F* and a multiplicative cocycle 
<7* over it, we consider a transfer operator 

C* : C°°(M) -> C°°(M), C*u(x) = g f {x) ■ u(F 4 (a;)). 

This is a one-parameter group of operators. In what follows, we consider the transfer 
operator £* associated to a C°° contact Anosov flow F* : M — > M and a C°° 
multiplicative cocycle g 1 : M — > C. In Section 5, we will introduce a scale of 
Hilbert spaces H£ niso (M) with a parameter r > 0, adapted to the flow F*. Those 
Hilbert spaces satisfy 

H r (M) C ff[ niso (M) C ff" r (M) 

where H r (M) is the Sobolev space of order r on M. Our main result is 

Theorem 2.1. 7/i is sufficiently large, the transfer operator L l extends naturally 
to a bounded operator on H^ niso (M) for any r > 0. If r > is sufficiently large, 
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the essential spectral radius of £' : H£ niso (M) — > H£ niso (M) * s exactly A*, where A 
zs f/ie quantity defined by 



The conclusion of the theorem implies in particular that, if t and r are sufficiently 
large, the spectral set of the operator £' : H£ niso (M) — > Hl niso (M) on the outside 
of the disk \z\ < A* consists of discrete eigenvalues with finite multiplicities. (See 
[16] for implication of this conclusion on decay of correlations.) 

Note that, if F* is the geodesic flow of a closed surface with constant negative 
(= —1) curvature and g l = 1, we have A = e -1 / 2 and the bound A* = e~*/ 2 on 
the essential spectral radius of £* in the theorem above is exactly the optimal one 
expected from the classical result of Selberg[ll] and the heuristic relation between 
the zeros of dynamical zeta functions and the spectrum of the generator of £*. 
(See also [12, 13].) We expect that the bound in the theorem above is optimal 
as far as we consider the action of £} on Banach spaces B such that C°° (M) C 
B C (C°° (M))' . Note that the peripheral eigenvalues outside the essential spectral 
radius is essentially independent of the choice of function spaces. (See [3, Appendix 



In the following sections, we proceed as follows. In Section 3, we set up a Darboux 
coordinate system on M and reduce the main theorem to the corresponding claim 
(Theorem 3.3) about transfer operators on the Euclidean space R 2d+1 equipped 
with a standard contact form oeo. In Section 4, we discuss about the FBI transform 
on the Euclidean space M. 2d . The FBI transform decomposes functions on K 2d into 
Gaussian wave packets parametrized by points in the cotangent bundle T*M. 2d = 
R 2d ©R 2d and was used in semi-classical analysis, by Sj6strand[15] and Martinez [10], 
in order to study microlocal properties of functions. In Section 5, we introduce the 
partial FBI transform on the Euclidean space R 2d+1 , which is roughly a combination 
of the Fourier transform in the direction of the flow and the (scaled) FBI transform 
in the transversal directions. We then define the anisotropic Sobolev space H^ niso by 
using the partial FBI transform and a weight function W£ niso . In the last subsection, 
we also study the action of a linear transformation with some hyperbolic property 
on the anisotropic Sobolev space -ff^niso an d prove an analogue of Theorem 3.3 in 
this simple case. In Section 6, we will decompose the transfer operator on the 
Euclidean space R 2d+1 into three parts, the compact, hyperbolic and central part. 
The compact part concerns the wave packets with low frequency and is a compact 
operator as its name indicates. The hyperbolic part concerns the wave packets that 
have high frequency in the transversal directions to the flow. In Section 7, we will 
show that the operator norm of the hyperbolic part is bounded by an arbitrarily 
small constant if we take a large parameter r in the definition of H^ niso . Thus the 
central part turns out to be most essential for our argument. We study the central 
part in Section 8. We will decompose the central part into small pieces so that each 
piece can be well approximated by the simple operator studied in the last subsection 
of Section 5. In Section 9, we give a lower bound for the essential spectral radius 
that coincides with the upper bound, finishing the proof of the main theorem. 




A].) 



Acknowledgement. The author expresses his gratitude to the Mittag-Leffler in- 
stitute and Prof. M. Benedicks (KTH) for hospitality during his stay at the institute 



4 



MASATO TSUJII 
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him about basic facts about the FBI transform and the ingenious change of coor- 
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3. Local properties of contact Anosov flow 

3.1. Darboux local coordinates. Let F t : M — > M be a contact Anosov flow 
and a the contact form on M preserved by the flow. By multiplying a by some 
smooth function, we may and do suppose that a(V) = 1 for the generating vector 
field V of the flow F*. 

On the 2d+l dimensional Euclidean space R 2d+1 , the standard contact form ao 
is defined by 

I " \ 

(1) ao = dx + \y~]xj ■ dxd+j - x d+j ■ dxj . 

\j=i J 
Note that the vector field d xo := d/dx is characterized by the conditions 

(2) a (d Xo ) = 1, da (d Xo , •) = 0. 
For a real number 9 > 0, we consider the cones 

C+(0) = {x = (x Q ,x+,x-) € R 2d+1 | < e\\x+\\}, 

C_(0) = {x = (x Q ,x + ,x-) e M 2d+1 | ||x+|| < 0||a;-||} 

where || • || is the Euclidean norm and, for x = (£i) 2 = *= R 2d+1 , we set 

x + = (xi, x 2 , ■ ■ ■ , x d ), x~ = (x d +i,x d+ 2, • • • , X 2 d)- 

Definition 3.1. For A > 1, a C°° diffeomorphism F : U -> U' = F(U) is said to 
be a A-hyperbolic contact diffeomorphism if it satisfies the following conditions: 
(HI) U and U' are open subsets contained in the unit disk in R 2d+1 , 

(H2) F preserves the standard contact form ao, and 

(H3) F is hyperbolic in the sense that 

DF(R 2d+1 \ C_(l/10)) C C+(l/10), 

DF-\R 2d+1 \ C+(l/10)) C C_(l/10) 

and that 

^^(Trt^))!! > A||7r t (w)|| for v G M 2d+1 \ C_(l/10), 

||Dif- 1 (7r t (w))|| > A||7r t (w)|| for v e R 2d+1 \ C+(l/10) 

where 7r t : R 2d+1 -> {0} © R 2d denotes the orthogonal projection to the 
components other than the first one. 

From Darboux theorem for contact structure[l, pp.168], it follows 

Lemma 3.2 ([16, Proposition 2.2]). There exist a finite system of coordinate charts 

{^■.V^U^^dR 2 ^ 1 }^ 
on M and a constant c > such that, for 1 < i,j < I, 
(1) a = K*(a Q ) on Vi and d XQ = (k»)*(V) on [/;. 
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(2) For sufficiently large t > 0, the diffeomorphism induced on the local charts 

4 := kj o F* o R7 1 : U*. -+ 4 ^ere E/£ := n t {V t fl 

is a c - X -hyperbolic contact diffeomorphism, provided U*j ^ 0, 
w/iere A > 1 is f/ie constant in the definition of Anosov flow. 

We henceforth fix a system of coordinate charts as above and define a family 
of transfer operators on the local charts as follows. Take functions pi £ C£°(Ui), 
1 < i < t, so that the family {p^ o/tj of functions on M is a C°° partition of 
unity subordinate to the open covering {Vi} and take another family of functions 
pi £ Cfi°(Ui) so that Pi(x) £ [0, 1] and that f>i(x) = 1 on the support of pi. The 
transfer operators L\- : C°°(Uj) -> C$°(Ui) for 1 < i, j < i and t £ M is then 
defined by 

£ t ij u(x)=g t ij (x)-u{Ff j (x)) 

where 

g t i j {x)=pi{x)-p j {Fl j {x))-g\KT\x)). 
These transfer operators as a whole form the operator 

L * : cS?(Ui) -»• Co 00 ^), L*(K)f =1 ) = ( 

/ i=l 

By the definitions above, we have the commutative diagram of operators: 



(3) 



C°°(M) £t > C°°(M) 



where 

t : C°°(M) -^0^), i(u) = ( W ■ uo/tr 1 )^. 



i=l 

In Section 5, we will introduce a scale of Hilbert spaces (7J^ niso , || • || r ) for r £ M. 
such that 

H r C ff: niso C b-\ 
where H T is the Sobolev space of order r on R 2d+1 . And we will prove 

Theorem 3.3. For any given r > d, there exists a constant Co > such that, for 
any X-hyperbolic contact diffeomorphism F : U — > V with X sufficiently large and 
for any g £ Cq°(U), the transfer operator 

(4) Cu = g ■ (u o F) 

extends naturally to a bounded operator £ : H^ niso — > H^ niso and the essential 
spectral radius of the extension is bounded by 

(5) Co ■ max{A(F, 5 ), Wg^ ■ X~ r ■ A(F,g)}, 
where A(F,g) and A(F,g) are defined as 

!<?(*)! \ 



A(F,g) = max 



6 su pp(s) \ v / \dct(DF x \ E +) 
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and 

A(F,g)= max y/\dct(DF x \ E+ )\, 

xesupp(g) 

with setting E+ = {(x , x+, x~) <= R 2d+1 \ x~ = 0}. 

For a bounded linear operator L : B — > B on a Banach space B, its essential 
operator norm ||£|| ess and essential spectral radius p ess (L) are respectively the in- 
fimum of its operator norm and its spectral radius under perturbation by compact 
operators, that is, 

||L|j 0SS := inf{||L — K\\ \ K : B — »■ B is a compact operator.}, 

Pess(L) := inf{p(L — K) \ K : B — > _B is a compact operator.} 

By definition the essential spectral radius of a bounded linear operator does not 
exceed its essential operator norm. 

We show that the claims of the main theorem except for the lower bound for the 
essential spectral radius follow from Theorem 3.3. Consider the (unique) norm on 
C°°(M) such that the injection i is an isometric injection into © i=1 -ff[ n i so with 
respect to it and let H£ niso (M) be the completion of C°°(M) with respect to that 
norm. From the former claim of Theorem 3.3, the commutative diagram (3) extends 
naturally to 

® e TT r L * s £D/ fir 

i=l -"aniso f \Ui=l n aniso 

l t 

H r ani jM) K n{so {M) 
in which t is an isometric embedding. From the latter claim of Theorem 3.3, we see 
that the essential operator norm of L* : © i=1 H^ niso — > i=1 H^ niso is bounded by 

(6) Co ■ max (max{A(i*, <&)> llffylU ■ Ao^ ■ A(i*,<&)}) 

and so is that of £* : H r aniso {M) -> iJ^ niso (M). Note that (6) is bounded by 

C • max J max - lligl] = , ||#* II • A„ rt • max JdetDF*\ Eu \ 
[xeM v /det^k * eM J 

with possibly different constant Co and that the latter term in max}-} above is 
smaller than the former if r is sufficiently large. Therefore, by multiplicative prop- 
erty of essential spectral radius, we conclude the estimate 

p ess (£*|^ niso(M) ) = lim \\£ nt : H£ ni (M) -+ H^{M)\\ V" < A*. 

3.2. Affine transformations and diffeomorphisms preserving the standard 
contact form a . For each point c = (c , c+, c") e R 2d+1 , let A c : R 2d+1 — > M 2d+1 
be the affine transformation 

(7) A c (x , x + , a; - ) = (x + c — c + • x~ + cT ■ x + , £ + + c + , x~ + c~) , 

which preserves the standard contact form ao- The totality .4 = {A c } ceR 2d+i of 
such transformations form a transformation group that acts on R 2d+1 transitively. 
In the following, most of our constructions (including that of the anisotropic Sobolcv 
spaces) will be invariant with respect to the action of the transformation group A. 

Let F : U — )■ U' be a C°° diffeomorphism between open subsets in R 2d+1 pre- 
serving the standard contact form a . Then it preserves also the vector field d XQ 
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as it is characterized by the conditions (2). Thereby, setting x\ = (xi,X2, ■ ■ ■ ,X2d) 
for x = (xo, x\, . . . , X2d), we may write F locally as 

(8) F(x Q ,x^ = (x + f(x^,F^)), 

where F t : M 2d -> R 2d and / : M 2d -> K arc a C°° diffcomorphism and a C°° func- 
tion respectively. Let af and Wf be the differential forms on K 2d defined respectively 
by 

Id \ i d 

(9) af — I Xj ■ dx J+d - Xj+d ■ dxj J , LJ-f = -da t = (fa, A dx d +j. 

\j i / " ./ i 

Then F-f above preserves the symplectic form and the function / is determined 
from F by the relation 

df = DF*(dx a ) - dx = a t - Ff(a t ) 

up to difference by a constant. 

Suppose F(0) — in addition. Then, for any 1 < i,j < 2d, we have 

These relations (in particular the latter) are not very obvious but the proof is 
straightforward. We refer [16, Lemma 4.1] for the detail. The property (10) implies 
that there exits a constant C > such that, for z € R 2d+1 sufficiently close to the 
origin and for £ = (£o,£ + >£~) € K 2d+1 , one has the estimate 

W'DF^) *£Fo(0|| < W'DfMW + nDFMZt) '(DFMmi 

<c(M-\\z\\ 2 +\m\-\\z\\). 

The last estimate is applicable to the germ of F at each point, by means of changes 
of coordinates by affine transformations in A. Thus we obtain the following propo- 
sition, which stands without the assumption F(0) = 0. 

Proposition 3.4. If a C°° diffeomorphism F : U — > U' between open subsets in 
R 2d+1 preserves the standard contact form oto and if K is a compact subset of U , 
there exists a constant C > such that 

rDFy® - 'DFy^m < cm ■ \\ V] - v \\\ 2 + u & ■ mhvM ■ ibt - v\\\) 

for any two points y = (y , y^),y' = {y' Q ,y\) in K and any £ = (£o,£f) e K 2d+1 . 

4. The FBI transform 

In this section, we introduce the FBI (Fourier-Bros-Iaglonitzer) transform and 
give a few basic facts related to it. As the Fourier transform decomposes functions 
into simple wave functions, the FBI transform decomposes functions into Gaussian 
wave packets. We refer [10, Ch.3] for general argument on the FBI transform. 

4.1. FBI transform. For a pair (x, £) of points x and 1 ) £ in the D-dimensional 
Euclidean space R D , we consider a C°° function 

cj> Xt£ : M. D — > C, 4> Xi t(y) = a D • exp (i£(y - (x/2)) - \\y - x\\ 2 /2) 

where a D = (2ir)- D / 2 ■ ir- D /\ 



Maybe it is more natural to regard £ as an element of the dual space of R D . 
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Remark 4.1. In the textbook [10], the function (f> x .£ is defined similarly but with 
the term i£(y — (x/2)) replaced by i£(y — x). Our definition above is slightly more 
convenient for our argument, though the difference is not essential. 

The FBI transform T maps a function u(x) on R D to the function 

Tu(x, o = J <j> x ,t(y) ■ u (y) d v 

on R 2D = R D © R D . Its (formal) adjoint T* maps a function v(x, £) on R 2D to the 
function 

T*v(y) = J <j) x ^(y) ■ v(x, £)dxd£ 

on R D . The following are the basic properties of these transforms. 

Proposition 4.2. (1) The FBI transform T is a continuous linear operator from 
S{R D ) to S(R 2D ), while T* is a continuous linear operator from S(R 2D ) to S(R D ), 
where S(R D ) denotes the Schwartz space on R D . 

(2) The composition T*°T : S(R D ) — > S(R D ) is the identity operator. Consequently 
the FBI transform T extends to an isometry from L 2 (R D ) to L 2 (R 2D ). 

Proof. The first claim can be proved by a straightforward argument. Below we 
prove the second. The Schwartz kernel of the operator T* o T is 



Performing integration with respect to ^ and x in turn, we see that this equals 
a 2 D ■ (2tt) d • S(y-y') ■ J exp(-||y - x\\ 2 ) dx = a 2 D ■ (2n) D ■ n D / 2 S(y - y') = S(y - y'). 
Clearly this implies that T* o T is the identity operator. □ 

Note that, if we put v(x,£) = Tu(x,£) for u e ^(IR 13 ), the latter claim of the 
proposition above implies the following expression of u as a superposition of the 
wave packets 4> x ^{-)\ 

u (y) = T*v(y) = J v(x,£) ■ (j> x ,i{y) dxd£. 
4.2. The projection operator V. 

Proposition 4.3. The composition V := T o T* : S(R 2D ) -> S(R 2D ) extends to 
the orthogonal projection V : L 2 (R 2D ) — > L 2 (R 2D ) to the closed subspace 

T(L 2 (R D )) = {Tu G L 2 (R 2D ) | u G L 2 (R D )}. 

Proof. From the definition of V and Proposition 4.2, we have V = V* and VoV = V . 
It remains to prove that 

T(L 2 (R D )) = {u G L 2 (R 2D ) | Vu = u}. 

Hue L 2 (R 2D ) satisfies Vu = u, it belongs to T(L 2 (R D )) because u = Vu = 
T{T*u). Conversely, if u G T(L 2 {R D )), we can take v G L 2 (R D ) such that u = Tv 
and obtain Vu = T o T* o Tv = Tv = u from Proposition 4.2 (2). □ 
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The projection operator V above is an integral transform 
with the kernel 

(11) K P (x,i;x',Z') = J Myj-<t> x >,?(y)dy. 

If we perform the (Gaussian) integration in K-p{-), we see 



where fl : R 2D © R 2D -> R is the standard symplectic form on R 2D = R D © (R D )* 
defined by tl((x,£), (jc'.O) = a; ■ f - f ■ a;'. 

We may generalize the projection operator f to a slightly more general setting. 
Let bj be a symplectic form on an even dimensional Euclidean space E and suppose 
that it is compatible 2 -* with the Euclidean norm. Wc define the integral transform 
P w : S{E) S{E) by 

(12) P u u(z) = {2n)- D ' 2 J exp(-i • u(z, z')/2 -\\z- z'\\ 2 /A) ■ u(z') dz', 

replacing fi in the expression of V above by Li. Then we have 

Proposition 4.4. P u extends to an orthogonal projection operator in L 2 (E). 

Proof. We can check P u oP u = P u and P* — P u by a straightforward computation. 
(Or, one can introduce an orthogonal coordinate system on E in which Q, — lj.) □ 

4.3. The action of affine transformations. We are going to consider the action 
of affine transformations viewed through the FBI transform. Let B : R D — > R D be 
an affine transformation. We write its natural action on T*R D = R D © R D as 

B : R D © R D -> R D © R D , B(x, £) = (Bx, t (DB)~ 1 ^), 

where DB denotes the linear part of B. We consider the pull-back operator by B, 

L B : S{R D ) -> S(R D ), C B u{x) = u(B(x)). 

The corresponding action of B on the functions on T*R D = R D © R D is 3) 

C B : S(R d (SR d ) ^ S(R d (SR d ), C b u(x, f) = e -* B_1 (0K/2 ■ u(B{x, £))■ 

Lemma 4.5. If B : R D — > R D is an isometry, we have 

C B oT = ToC B , C b oT*=T*oC b VoC b = C b oV. 

Proof. The first and second equality follows from the relation ^■b4 > b(x() = < t )x ,i- 
The last follows from these and the definition V = T o T* ■ □ 



^Compatibility implies that the linear map J : E E defined by the relation ui(x, y) = (x, Jy) 
satisfies J o J = — Id. 

3 )The coefficient e (°)'?/ 2 appears as the result of our definition of (p Xt £. See Remark 4.1. 
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If the affine transformation B : R D — > M. D is not an isometry, the lemma above 
is no longer true and we need some modification. Below we assume that B is a 
linear transformation, i.e. B(Q) = and that dct B = 1, for simplicity. 

If we define the operator C B ■ S{R 2D ) -> S(R 2D ) by 

(13) E B = ToC B oT*, 

it makes the following diagram commutes: 



S(IR D eIR D ) Cb > S(R D R D ) 



(14) 



r 



T 



S(R D ) — S(R D ) 



So it should be natural to call Cb the lift of Cb with respect to the FBI transform. 
For the linear transformation B, we set 

d(B) =det((Id + 4 5-5)/2) 1/2 . 

Then we have the following expression for the lift C b ■ 

Proposition 4.6. C B = d(B) ■ V o Cb ° V . 

Proof. The operator T* ° Cb ° T can be written as an integral operator 
(T* o C B o 7>(y) = / #(y, y') u(y') <V 

with the kernel 

A-fcM,') = a 2 D ■ J e iC-{y-B- 1 y')-\v-x\'/2-\y'-Bx\'/2 dx ^ 

If we calculate the integration using a change of variable z = x — B~ 1 y', we obtain 
K(y, y') = n-°/ 2 ■ S(y - B" V) ■ / e -l*- V-^/a-lw'-**!^ 

= ^/ 2 • - B"V) / e-l z l 2 / 2 -l Bz l 2 / 2 dz = d(B)- 1 • ,5(2/' - By). 

This implies £b = d(-B) -T* ° Cb°T ■ Composing T and T* from the left and right 
respectively, we obtain the required formula. □ 

4.4. Change of variables. In this subsection, we set D — 2d and consider a 
linear transformation B : R 2d — > R 2d that preserves the symplectic form u>-\ on R 2d 
defined in (9). Below we show that the operator Cb can be identified with the 
tensor product of two (almost) identical operators through an appropriate change 
of variables. This argument is essentially due to F. Faure[5]. 

To begin with, note that the Hilbert space L 2 (R 2d ©R 2d ) is naturally identified 
with the tensor product L 2 (R 2d ) ® L 2 (R 2d ). Let us consider the linear bijection 

(15) Z : R 2d ® R 2d -> R 2d ® R 2d , Z(x, £) = (2~ 1/2 (£ + Jx), 2" 1/2 (^ - Jx)) 

as a coordinate change, where J : R 2d — > R 2d is the linear map characterized by 
the condition (y, Jx) — uj^(y,x) or, more concretely, defined by 

J(x + ,x~) = (x~,-x + ) for x + ,x~ e R d . 
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From the assumption that B preserves ojf, the diagram 

B 

> 



commutes. Consequently, for the unitary operators 

Z* : L 2 (R 2d © R 2d ) -> L 2 (R 2d © R 2d ), Z*u{x, f ) = u(Z(a;, 0), 

and 

C : L 2 (R 2d ) -+ L 2 (R 2d ), Z>(£) = u^S"^), 
the following diagram commutes: 



h is 



z* 



z* 



Another important property of Z is that it is an isometry with respect to the 
standard Euclidean norm on M. 2d © R 2d and intertwines the standard symplectic 
form f2 with © (— wj). That is, for Z(x, £) = Z(x',^') — (z',w'), we have 

and 

^((z, 0i £')) = w o(z, z') - w (tw, tt/). 

Let Po : L 2 (R 2d ) -> L 2 (R 2d ) be the projection operator defined by (12) with 
the setting D = d and oj = w-f. Then, from the property of Z noted in the last 
paragraph, we see that the following diagram also commutes: 



z* 



Z' 



L 2 (R 2d ®R 2d ) L 2 (R 2d © R 2d ) 

Therefore, if wc define the operator Co : L 2 {R 2d ) -> L 2 (R 2d ) by 

(16) Ca^diB) 1 / 2 -V o Co oV , 

we have the commutative diagram 

L 2 



(17) 



' 2,72-/ -<:2.l , C B > ^2 



Z" 



Z" 



v2d\ 



In conclusion, the operator Cb is identified with the tensor product of the operator 
Co and its complex conjugate through the coordinate change by Z. 
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5. Definition of anisotropic Sobolev spaces 

In this section, we introduce what we call the partial FBI transform and then 
give the definition of anisotropic Sobolev spaces using it. 

5.1. The partial FBI transform. The partial FBI transform on R 2d+1 is a com- 
bination of the Fourier transform in the first coordinate and the FBI transform in 
the other coordinates with some scaling. Below we give a precise definition for it. 
Take and fix a C°° function x ■ K -> [0, 1] such that 



X(s) 



1, ifs<4/3; 
0, ifs>5/3. 

For a real number s, we set 4 ' 

<*> = M-(i-x(M)) + x(M), 

so that (s) > 1 for any s and that (s) = \s\ holds if \s\ > 2. For a given point 
V = (y»)i=o e K 2d+1 , we will write 

V + = (2/1,2/2, •■• ,Vd), V~ = {Vd+\,y n +2, ■ ■ ■ ,V2d) and y^ = {y + ,y~). 
As in the definition of the FBI transform, we introduce a family of functions 
$* t ,£ : K 2d+1 -> C for (x h f ) G M. 2d © R 2d+1 

defined by 

*-t.«(tf) = ^y/T ' ex P (^02/0 + t£ t (l/t - N/2)) - (Co)lbt - ^tH 2 /2) 
^ L_ . e «oyo^ . ^(^ ) d /2 . a2d . e ^tfet-(*t/2))-<eo}||?/t-*tll 2 /2^ _ 

The partial FBI transform & maps a function u(y) on M 2 <2+i ^ ^he function 



&u(xi,Z) = J* x ^(y)-u(y)dy on R 2d ®M 2d+1 . 
And its formal adjoint maps a function uf^f, £) on R 2d © IR 2<i+1 to tne f unc tion 
3"u(y) = J$ X t, ( (y)u(x h Odx t dZ on R 2d+1 . 

The partial FBI transform may be viewed as a combination of the Fourier trans- 
form and the FBI transform with some scaling as follows: Consider the Fourier 
transform in the first variable, 

^0 : S(R 2d+1 ) -+ S(R 2d+1 ), & u{£o, 2/t) = (^V 2 J e-*>">u(yo, y^dy 
and the FBI transform in the other coordinates, 

T t :S(R 2d+1 )^S(R 2d (BR 2d+1 ), T^fa,® = /^t)'^o^f)^ t , 
where £ = (£ , £t) e K2d+1 = K © K2d - Also , consider the operators 

5 : S(R 2d+1 ) -> 5(M 2d+1 ), S«Ko,a:t) = (Co)- d/2 ■ (Co>- 1/2 ^t) 



4 )l n most of the literature, (s) is defined to be (1 + s 2 ) 1 / 2 . But the definition here is more 
convenient for our argument. 
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and 

S : S(R 2d ®R 2d+1 ) -+ S(R 2d ffiR 2d+1 ), Su(x h Q = u ({^)- 1/2 x h (£„, (£ } 1/2 £ t ) 
associated to the scaling 

R®R 2d 9 (fr.st) ^ (£o,(£o)~ 1/2 z t ) £lffil M 
Then the FBI transform 2T and its formal adjoint £?* are expressed as 
ST = (S- 1 oT t oS)o JF„, ,J* = o (S- 1 o T* o S). 
From this expression and the properties of the Fourier and FBI transform, we obtain 

Proposition 5.1. (1) The partial FBI transform ST is a continuous linear operator 
fromS(R 2d+1 ) to S{R 2d ®R 2d+1 ), while its formal adjoint ,T* is a continuous linear 
operator from S{R 2d R 2d+1 ) to S(R 2d+1 ). 

(2) The composition ,<7* o ST : S(R 2d+1 ) -> S(R 2d+1 ) is the identity operator. 
Consequently 2? extends to an isometric embedding of L 2 (R 2d+1 ) into L 2 (R 2d © 



Proposition 5.2. The composition £?> := ST o 5^* : <S(R 2d © R 2d+1 ) -> 
R 2d+1 ) extends to the orthogonal projection 3? : L 2 (M. 2d ®R 2d+1 ) -> L 2 (]R 2d ffi]R 2d+1 ) 
to t/ie dosed subspace ,9 \L 2 ; (R 2d+1 j) = {^u | u e L 2 (R 2d+1 )}. 

5.2. Anisotropic Sobolev spaces. In this subsection, we define the anisotropic 
Sobolev space H£ niso as the pull-back of some weighted L 2 space on R 2d © R 2d+1 
by the partial FBI transform 07 . 

For 6 > 0, let C;(6») and C* (0) be the cones in R 2d defined by 

C* + (0) = {((+,(-) tm. 2d = R d ®R d \ \\C\\ <0\\( + \\}, and 

C*_(6) = {{(+,C) tR 2d = R d ®R d \ ||C + || <0||C~l|}- 

We take C°° functions Vv : PR 2d -> [0, 1], = ±, on the projective space PR 2d 
such that 



V+([C])+V'-([C]) = 1 and 



V+([C]) = i ifCec;(i/3), 
V-([C]) = i ifCeC*_(i/3), 



where [(] denotes the element in PR 2d that is represented by £ € R 2d , and then 
introduce the function 

W: niso : R 2d R, VF; niso (C) = lM[C]) ' (IKII)- r + V>-([C]) • (IICII) +r . 
We define the weight function W'[ niso : R 2d © R 2d+1 -> R+ as follows: In the case 
x = 0, we set 

wi niso (o, o = vF a 2 : iso (^|i7i) for e = (Co, e t ) e R 2d+1 = r « 

(Notice that we have 2r in the superscript of W 2 ^ iso (-) on the right hand side.) 
Then we extend this definition to the case x ^ uniquely so that it is invariant 
with respect to the natural action of the transformation group A. In other words, 
we set 

wLuoto.O = w^(o,*iW(o,*t)(0)- 

Note that ^4(o,x t ) is defined in (7) with setting c = (0,a; t ) e R 2d+1 , so that 

'da^^o, e t ) - (6 , e + + Co • x- , r - e • * + ) = ( e t + • j(ao ). 
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Now we define the anisotropic Sobolev space Hl niso as the completion of the 
Schwartz space <S(R 2d+1 ) with respect to the norm 

\\u\\ r = HWaniso ' ^u|| i 2( R 2d ffiR 2d+l ) • 

By definition, the partial FBI transform & extends to the isometric embedding 

ST : ^ niso -+ L 2 (R 2d e R 2d+1 ; w: niso ) 

where L 2 (R 2d ® R 2d+1 ; W£ niso ) denotes the weighted L 2 space with weight W^ niso . 

We give a relation between the anisotropic Sobolev spaces introduced above and 
the usual Sobolev spaces. (The proofs of Lemma 5.3 and Corollary 5.4 below will 
be given in the appendix.) Recall that the Sobolev space H r of order r on R 2d+1 is 
defined as the completion of the Schwartz space <S(R 2d+1 ) with respect to the norm 

h\\ H r = \\(O r -^(ti)\\L^ + l) 

where & denotes the Fourier transform. For another norm 

IHl'm = II (0 r • & II L^B^ei^+i) 
defined by using the partial FBI transform 3*, one can show 

Lemma 5.3. The two norms \\ ■ \\h^ and \\ ■ \\' Hr on <S(R 2d+1 ) are equivalent. 

For a subset K C R 2d+1 , let C°°{K) be the set of C°° functions whose supports 
are contained in K, and let H^ niso (K) (resp. H r (K)) be the closure of C°°(K) in 
-^aniso ( res P- H r ). As a consequence of the last lemma, one obtains 

Corollary 5.4. For any compact subset K C M. 2d+1 , we have 

H r (K) C H r aniso (K) C H~ r (K). 

5.3. The action of linear transformations on H^„-. Let us consider a linear 
transformation 

Id B : R 2d+1 -> R 2d+1 , (Id B){xq, x t ) = (x , B(jc t )) 

where B : M. 2d — > M. 2d is a linear transformation satisfying the following hyperbol- 
icity conditions for some large A > 1: 

(Bl) B(R 2d \ C* (1/10)) C C^l/lO), B- 1 ^ \ C^l/lO)) C C* (1/10), 

(B2) \\B(v)\\ > X\\v\\ if v e R 2d \ C* (1/10), and 

(B3) H-B- 1 ^)!! > A||w|| if v e R 2d \ 0^.(1/10). 
Below we study the pull-back operator 

Adesw = u o (Id B) 

acting on the anisotropic Sobolev space Hl niso7 as a simple model of the transfer 
operator (4). To this end, we introduce the operator 

Ad©B = & ° Nicies ° ^* 
which makes the following diagram commutes: 

S(R 2d ®R 2d+1 ) S(R 2d (S)R 2d+1 ) 

sr 

5(R2d+l) S(R 2d+1 ) 



sr 
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In order to see that the operator £id©s induces a bounded operator on H^ niso , it is 
enough to check that the lift £id®B extends to a bounded operator on the weighted 
L 2 space L 2 {R 2d © R 2d+1 ; W r aniso ) with the norm ||u|| := ||W^ niso • u|| L 2. 

Recall that the partial FBI transform is a combination of the Fourier transform 
in the flow direction and the FBI transform with some scaling in the transversal 
directions. Since the map Id © B preserves the frequency in the flow direction, we 
can separate the actions of £id©B into each frequency. Thus, taking the scaling 
in the transversal direction into account, we see that the operator norm of the lift 
Ades on L 2 (R 2d © R 2d+1 ; W^niso) equals the supremum of the operator norms of 



aniso/ 

(18) C B ■ L 2 (R 2d ©R 2d ;W (?o) ) -> L 2 (R 2d © R 2d ; W (€o> ) 
for £ € R, where C B is defined in (13) and W s : R 2d © R 2d -> R is defined by 

(19) W.{x h b) = W? ( ?t + J:Et 



2r 

aniso 



.<(i + *- 1 -||£t + ^tll 2 ) 1/2 ) 1/2 , 

Next recall the change of variables discussed in Subsection 4.4, in particular, the 
commutative diagram (17). Since we have 

(20) W s o Z-\z,w) = V s (z) := W £ iso ( ((1 + ^ , ^ ||2)1/2)1/2 ) , 
the operator (18) is identified with the tensor product of 

Co : i 2 (M 2d , V s ) -> L 2 (R 2d , V s ) and T : L 2 (R 2d ) -> L 2 (R 2d ) 
where £ is that defined in Subsection 4.4. And, for these two operators, we show 

Lemma 5.5. The operator Co extends naturally to bounded operators both on the 
Hilbert spaces L 2 {R 2d ) and L 2 (R 2d ,V s ) for s > 1. Further, 

(1) the operator norm of Co : L 2 (R 2d ) -> L 2 (M. 2d ) is 1, and 

(2) the operator norm of C Q : L 2 (R 2d , V s ) -> L 2 (R 2d , V s ) is bounded by 

Co ■ max{d(B)- 1 / 2 , d(B) 1 / 2 • A- r } 
where Co is a constant that does not depend on B nor s > 1 . 

Once we prove this lemma, we conclude that the operator norm of Ci^b on 
H r aniso is bounded by Co ■ max{d(B)- 1 / 2 , d(B) 1 / 2 ■ A" r }. Notice that the last quan- 
tity corresponds to (5) in Theorem 3.3 since d(B) is proportional to det . In 
the following sections, we will prove Theorem 3.3 by reducing it to this simple case. 

Proof of Lemma 5.5. The claim (1) follows, for instance, from the fact that the 
operator norm of C a © C on L 2 (R 2d © R 2d ) equals that of C B on L 2 (R 2d © R 2d ) 
and hence equals 1, as we showed in Subsection 4.4. Below we prove the claim (2). 
We will use Co as a generic symbol for constants that do not depend on B nor s. 
From the definition, the operator Co can be written as an integral operator 



Cou(z) — J K{z, z')u(z') dz' 

and the kernel satisfies 

\K(z, z')\ < (2ir)- d ■ d(B) 1 / 2 ■ J exp(-p - w\\ 2 /A - fB^w - z'\\ 2 /4)dw. 
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Hence, to prove the claim (2), it is enough to show 

|| Q o C o Q : L 2 (R 2d , V s ) -> L 2 (R 2d , V s )\\ < C ■ mzx{d(B)-\ X~ r } 
where Q is the convolution operator 

Qu(z) — (j) * u(z) with 4>{z) — exp(-||z|| 2 /4). 
By using the rapidly decaying property of cj> and the definition of V s , we have 

(21) V.(z) ■ (f>(z - z') < Co ■ V s (z') .(\\z- z'\\)- 2d - 1 

where Co is a constant that does not depend on s, provided that we take appropriate 
functions ip± in the definition of W* niso (-). (See Remark 5.6 below.) This estimate 
implies in particular 

||Q : L 2 (R 2d ,V s ) -> L 2 (R 2d ,V s )|| < C . 

Remark 5.6. To have the inequality (21) hold, it may be necessary to put a technical 
condition on the functions ip± to avoid pathological cases, though we do not know 
whether it is really necessary. For instance, if we assume the condition that the 
first derivatives of the functions £ i-> (ip±) e ([£]) is bounded on the unit sphere for 
each e > 0, which can be fulfilled easily, we can show that the first derivatives of 
log V s is bounded by a constant independent of s > 1 and hence the inequality (21) 
follows. 

Let £ be the ellipsoid in R 2d defined by the condition 
\*z ■ (I + B^ 1 ■ t B~ 1 )~ 1 • z\ < 1. 
From the definition of V s and hyperbolicity of B, it holds 

VsiB'z) < Co ■ A~ r • V.(z) for z <£ £ 
and therefore we have 

(22) || Q o £ o(Id - l f ) o Q : L 2 (R 2d , V s ) -> L 2 (R 2d , V s )\\ 

< Co\\£o o (Id - l e ) : L 2 (M 2d , V a ) -»• L 2 (R 2d , V s )|| < Co ■ \- r 

where \$ denotes the multiplication by the characteristic function of £ . 
The operator norm of the remainder part 

(23) Q o Co o l f o Q : L 2 {R 2d , V s ) i 2 (R 2d , V s ) 
equals that of the integral operator 

£' : L 2 (R 2d ) -> L 2 (M 2d ), £'u(z) = ^ fc(z, «') u(z') d?' 

with the kernel 

fc(z, z') = f ■ <p(z - 'Bw) ■ <P(w - z') dw. 

Once we prove the estimates 

(24) sup J k(z,z')dz < C ■ d(B)- 1 and sup J k{z, z 1 ) dz' < C ■ d(B)-\ 

the Schur test [8, Lemma 18.1.12] will yields the estimate that the operator norm 
of £ is bounded by Co • d(B)^ 1 and so is the operator norm in (23), which together 
with (22) completes the proof of the claim (2). 
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To finish the proof, we prove (24). From hyperbolicity of B and the definition 
of the function V s (-), it is not difficult to check 

/ Vi S B ^ ■ 4( w - z ') dw<C - d(B)- 1 for ally's 
Je Vs(z) 



and 

Vs ^ <P{z -w)dw<C Q - d(B)- 1 for all z e ™ 



Jb; 



B£ V a (*B-%) 

By virtue of (21), the former inequality above implies the former claim in (24): 

Similarly the latter inequality above implies the latter claim in (24). □ 

6. Transfer operators on R 2d+1 

6.1. Transfer operators and their lifts. In this section and the following, we 
consider in the setting of Theorem 3.3: Let F : U —¥ U' be a A-hyperbolic contact 
diffeomorphism with large A ^> 1 and g : M 2d+1 -^Ca C°° function whose support 
is contained in U ; And we consider the transfer operator 

C = C F . g : C^(U') -> C °°(C/), Cu(x) = g(x) ■ u(F(x)). 

We define the lift of C with respect the partial FBI transform 3 by 

Z := ST o L o ST* : S(R 2d 8 R 2d+1 ) -> S(R 2d 8 K 2d+1 ). 

Then, as (14) in the case of FBI transform, the following diagram commutes: 

S(R 2d 8 M 2d+1 ) — ^— > S(R 2d M 2d+1 ) 



sr 



sr 



C?(U') — ^ C§?(U) 



The operator C is an integral operator 

Cu(x\,£) = J K(xj,£;zj,r])u(zj,Ti)dzjdii 

with the kernel 



(25) K(x h ti;z h r ] )= I * Xi ,t(y) ■ g(y) ■ *z uv {F(y)) dy 

Note that K(x^,£; Zj,rj) is bounded in absolute value by 



2rr 



J g(y) . e -HvF(v)-Zv) . e -(io)\\x f -y f \\ 2 /2-( no )\\F f (y i )~z i \\ 2 /2 dy 



Applying integration by parts to the integral above, we obtain the following crude 
estimate on the kernel K(-) of C. 

Lemma 6.1. For any p > 0, there exits a constant C p > 0, which depends also on 
F and g, such that 



(26) \K(x f ^;z f ,r,)\<C p -(^) d / 2 (r, ) d / 2 - f K (x h £ z h m y\ 



"dy 
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where 



(27) n(xt,t,w,v) - (b-vo) {-J^Tjr) \ (r?o) -i/ 2 / { {M) i/2 



Proof. Consider the differential operators 
1 + i(vo - Co) • d yo 



Ln = 



l + (%-Co) 2 ' 



Lx = 



i + WWFyin) -cil 2 ' 



Since 



L (e~ l{vF(y) -ty)^ = Li (e-i^FM-ty)^ 
we obtain, by integration by parts, that 

_4 d -^) d/2 -(m) d/2 



i(vF(y)-iy) 



\K{x h ^z h r])\ = 



2tt 



J K v y (x h ^z h rj;y)dy 



for any integers v, v' > 0, where 

K v y(x h Z;z hV ;v) = (LI)"' (L* o y ( e -«o>l|xt-»tll 2 /2-^>l|F t(l , t )-« t f /2 . 

The calculation in the definition of K u y{-) above is not simple, but one can check 
the following estimate by an inductive argument on v and v' . (See also Remark 6.2 
below.): For any integers it,tt' > 0, there exists a constant C > 0, which depends 
also on F and 5, such that 
(28) 

\K'(xi,S; z hV ;y)\<C- T\(x^,y, Co)" ■ T 2 (y, z h • T 3 (Co, 770)^' ■ T 4 (C, »?, yY 



where y = (2/0, 2/f) and 
7i(a;t,j/,Co) 



1 



2MCo,%) = 



«$>> 1/2 -H*t-»tl 
1 



(\\v\\) 1/2 (Nl) 



<co-%>' — (II^M-CII) <uwur7)-cu> 2 ' 

Remark 6.2. In deriving the estimate (28), use the fact that 

9« ^ e -(eo)|k t -J /t || 2 /2-{ t) o>||F t ( ! y t )- Zt || 2 /2 . ^(y^ 

< C a ,^ • max{(£ ), (r/ )} H/2 ■ T^, y, Co)" ■ T 2 (y, z^m f 
for any multi-index a and any /j, \x' > 0, and that (Co) < (??o) + (Co — Vo)- 
In the case where (||??||) 1/ ' 2 < (l^DFy^rj) — C||) holds, we have 

4 



(VDFyW-zw/iWnWm 



and, hence, the inequality (26) follows from (28). Otherwise, the inequality (26) 
follows again from the same estimate (28) but with setting v' = 0. □ 
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6.2. Decomposition of transfer operators. Our next task is to decompose the 
transfer operator £ and its lift £ into three parts, namely, the compact, hyperbolic 
and central part. The decomposition depends on two constants. One of the con- 
stants is t > 1/2. We take r close to 1/2 according to d and r. For instance, it is 
quite enough to assume 

1 1 1 

- < r < — I . 

2 2 100 • r ■ d 

The other constant is TV > 0. We will choose TV as a large constant in the course 
of the argument below so that several claims hold true. Note that the choice of N 
will depend on F, g and r. 
We define two function 

X : R 2d 8 R 2d+1 -> [0, 1] and X ctr : R 2d © R 2d+1 -> [0, 1] 

as follows: Recall the function \ introduced in the beginning of Section 5. In the 
case where x = 0, we set 

X {Q,0=Xm\\/N) and X ctr (0,£)=X 

for £ = (£o, £f) € R 2d+1 . Then we extend these definitions to the case x ^ uniquely 
so that they are invariant with respect to the natural action of the transformation 
group A on R 2d © R 2d+1 . More concretely, we set 

*o(* t ,o = xm\ 2 + ne - Co • ao(*t)ii 2 ) 1/2 M0 

and 

Y , t N f jlj-jg -«o(^t)ll \ 

Note that the support of Xo is contained in the neighborhood 

{(x t ,e)GK 2d ©R 2d+1 | \Co\ 2 + U-Co-a (x f )\\ 2 <(2N) 2 } 

of the zero section, while that of X ctv is contained in the neighborhood 

{(x f ,0eR 2d ®R 2d+1 | U-^-a (x f )\\ <2.(£o) T } 

of the one-dimensional subbundle spanned by a n . 

We define the compact, central and hyperbolic part of the lift £ respectively by 

£ cpt : S(R 2d 8 R 2d+1 ) -»• 5(M 2d © M 2d+1 ), £ cpt u = £(X • u), 

£ ctr :S{R 2d ®R 2d+1 )^S(R 2d S)R 2d+1 ), £ ctr u = £(X ctr • (1 - X ) • u), and 

£ hyp : S(R 2d 8 R 2d+1 ) -> 5(M 2d © M 2d+1 ), £ hypM = £((1 - X ctT ) • (1 - X ) ■ u). 

Clearly the lift £ is decomposed into these three operators: 

£ £cpt "f" £ctr "f" £hyp- 

For the transfer operator £ itself, we define its compact, central and hyperbolic 
part respectively by 

£ ff : S(R 2d+1 ) -> 5(M 2d+1 ), £ ff = ^* o £ ff o ^ 

with cr = cpt, ctr, hyp, so that we have £ = £ cpt + £ c tr + £h yp - 
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6.3. The compact part. The compact part of the transfer operator L is in fact 
a compact operator, as the name indicates. 

Lemma 6.3. The compact part £ cpt extends to a bounded operator £ cpt : H^ niso — > 
-^Iniso- extension is a trace class operator and hence a compact operator. 

Proof. We can check that the operator £ cpt is an integral operator with smooth 
kernel and maps the Sobolev space H r into C°°(supp.g) continuously. Thus one 
may view the operator £ cpt : H^ niso — > -ff^niso as tne composition 

#aniso ^ S (SU PP5 ) — -i— > ^(suppff) > ff^ iao 

where s > r is an arbitrarily large number and t denotes the injections. Since 
the injection i : iJ s (suppg) — > _ff r (suppg) is a trace class operator if s — r is 
sufficiently large[17, Ch.10.2] and since the composition of a trace class operator 
with a bounded operator is again a trace class operator, we obtain the lemma. □ 

The hyperbolic and central part will be considered in the following two sections. 

7. The hyperbolic part 

In this section, we consider the hyperbolic part of the transfer operator. We will 
use the notation in the previous sections and set A hyp = (1 — X ) ■ (1 — X ctT ) for 
simplicity. From Lemma 6.1, we see that the action of C is closely related to the 
pull-back operator by the natural action 

F : R 2d © R 2d+1 -> R 2d © R 2d+1 , F(x h £) = (i^t). *^ F (o!x t )(0) 

of F on M. 2d R 2d+1 post-composed by the multiplication by g(xf). And, by the 
definition of the weight function W^ niso and hyperbolicity of F, we have 

W r aTliso (F-\x h 0) < Co • A" r • W: niso (x t , for {x h Q € su PP X hyp 

where Co > is an absolute constant. (In fact, the weight function W.[ n i so is 
designed so that this inequality holds.) In view of these observations, the claim of 
the next proposition should be a natural one. 

Proposition 7.1. The hyperbolic part £hyp extends naturally to the bounded oper- 
ator £h yP : #Iniso ~~ ► ^aniso- Further the operator norm of the extension is bounded 
by Co • | \g | |oo • A~ r , w/iere Co > is a constant independent of F and g. 

We give an elementary proof for this proposition in the following subsections. 
But, since the proposition is intuitively rather obvious as we observed above and 
since this is not a main point of our argument, one may skip the proof below and 
proceed to the next section where we treat the central part. 

7.1. A Littlewood-Paley type partition of unity. To begin with, we introduce 
a partition of unity on R 2d © R 2d+1 and then define a norm on Hl niso , which is 
equivalent to the original norm || • || r but more tractable for our purpose. 

First we consider a simple partition of unity {x„ [0,1] | n = 0,1,2,...} 

on the real line R defined by 



Xn(s) = 



X(M), ifn = 0; 

X (2-"| S |)-x(2-" +1 M), ifn>l, 
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where x is the function introduced in the beginning of Section 5. Using this partition 
of unity and recalling the functions tp± : PM. 2d — > [0,1] used in the definition 
of W^ niso , we define the C°° partition of unity {ib m : R 2d -> [0, l]} meZ on R 2d by 

\xm(m\)-^+m), ifm>0; 
lM£ t )=SXo(||£tll). ifm = 0; 

lxM(llftll)-V'-(Kt]). ifm<0. 
By this definition, there exists a constant Co > 1 such that 

c -i J- 2- 2 "™ • ^) 2 < ^ niso (^ t ) 2 < Co £ 2- 2 ™ • ^ m (£ t ) 2 for £ t e M 2d . 

Finally we define the C°° partition of unity {* m : R 2d ® M 2d+1 ->• [0, l]} mGZ on 
_1 as follows. In the case x-f = 0, we set 

*m(0,Z) = ipm( J|L) where Ko,^ t ), 



V(licil) 1 / 2 / 

and then extend this definition to the case i / uniquely so that it is invariant 
with respect to the natural action of the transformation group A. That is to say, 
we set 

The estimate on the function WJ • „ above implies that 

Co" 1 • ^ 2- 4 ™ • * m (x t ,C) 2 < WI niso (x t ,C) 2 < Co • Y, 2 ~ iTm ■ *™(^t^) 2 

for G M 2d ©IR 2rf+1 . Consequently the anisotropic Sobolev norm || • || r satisfies 

Cq- 1 • Y 2 ~ 4 ™ • II*™ • ^ni 2 < ii«ii? <Co-Y 2 ~ 4rm ■ II*™ • ^ni 2 - 

Therefore, in order to prove Proposition 7.1, it is enough to prove the same claim 
with the norm || • || r replaced by the new norm 

\ 1/2 

- 4rm ■ii* ro -^«ni a ) . 




7.2. The proof of Proposition 7.1. Below we give the proof of Proposition 7.1 
assuming a lemma (Lemma 7.2) whose proof is postponed until the next subsection. 
Take a function u £ S(M. 2d+1 ) arbitrarily and set 

v = £h yP u, u m = * m • &u, v m = * m • ,7v = * m • £ hyp o ,7u. 
As we noted at the end of the last subsection, it is enough to show the claim 

(29) y 2 ~ 4 ™ • \\ v ^ ^ c ° • ni~ • A ~ 2r • E 2 ~ 4rm • imi£- 

To proceed, we decompose v m into countably many pieces 

V m ,m' = *m ' £hyp(u m >), w! € Z. 

The claim (29) is a consequence of the following three facts on the relation between 
the 1? norms of v m , m ' an d u m >- The first is the fact that 

||Wm,m'|U 2 < IMU 00 ' ||Mm'|U 2 
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for any m, to' S Z. This follows from the facts that the operators £f and ST* do 
not increase the L 2 norm and that the operator norm of C with respect to the L 2 
norm is bounded by ||<7||l°°- The second is that, for any given N' > 0, we can take 
the constant A > in the definition of X so that v m ^ m > — whenever \m'\ < A'. 
In fact, if we take sufficiently large A > according to A', the condition |m'| < A' 
implies that suppAh yp fl supp 'I'm' = and hence that £h yp (?w) = 0. The third is 
stated in the following lemma: 

Lemma 7.2. For any v > 0, there exists a constant C v > 0, which may depend on 
F and g, such that, if 

(30) m<m' + (l/2)log 2 A-3 

then 

\\v m , m '\\L* < C v ■ max < m > m '}|K„,|| L 2. 

As we will see later, the assumption (30) is a sufficient condition for the image of 
supp by F^ 1 not to intersect supp * m and therefore the claim of Lemma 7.2 is 
quite natural. But, unfortunately, our proof of Lemma 7.2 is not very short. So we 
will give it in the next subsection and below we finish the proof of Proposition 7.1 
assuming Lemma 7.2. 

Take and fix v such that v > 2r. Also take large A' > and set 

'0 if|m'|<A'; 
1m,m> = I 2 2r ( m '- m ) • Halloo if |to'| > A' and to > to' + (1/2) log 2 A - 3; 

C v ■ 2-(^-4'')-max{ m ,m'} if | m /| > jy/ and m < m ' + (!/ 2 ) l g 2 A - 3. 

We may and do take large A' > so that 



(31) 



7T7 



,.' < Cn • 



■A _r , 



7m,m' 

< Co • II5II00 • a- 



where Co is a constant independent of i* 1 and g. (Take large A' > according to i* 1 
and g so that C„ • 2~( l/ ~ r ) Ar is small.) From Lemma 7.2 and the two facts stated 
in the paragraph preceding it, we have also 

(32) 2- 2rm \\v m , m ,\\ L 2 < 7m>ro , • 2~ 2rm '\\u m ,\\ L 2 for any (to, to') e Z © Z, 

provided that we take sufficiently large A according to A'. Now, by using Schwarz 
lemma, we obtain the required estimate (29): 

2 



E 2 ~ 



Arm 



>\v m \\ 2 L 2 = r 

mEZ 



Arm 



< 



E 2 ' 



Arm 



E 



777 



E %n,m' ll W 77l,m' || L 2 J J 

m'GZ / / 



< (CollffHoo • A"') • 2 £ 2- 4rm 7 m ^ • II v m , m .\\l> by (31) 



< (Collglloo-A- 1 -)- ]T E 2 



-Arm' 



I twill* b y ( 32 ) 



< (CollffHoc • A- 1 ") 2 • £ 2- 4 ™'|K„,||i 2 by (31). 



This completes the proof of Proposition 7.1. 
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7.3. Consequences of the condition (30). Before we go into the proof of Lemma 
7.2, we give consequences of the assumption (30) in the lemma. 

Lemma 7.3. There exist a small constant c > and a large constant C > 0, which 
depend on F, such that, if m and m! satisfy (30) and \m'\ > C, and if 

V = (2/0, J/f) € suppg, (y t ,0 € supp* m , and (.F t (j/ t ),j/) € supp* TO -, 

(33) - ^W)!! > c • 2 m ^l™l'l-'l> • max{(||e'||) 1/2 , (h'||) 1/2 }. 

Proof. Since the function ty m is invariant with respect to the natural action of A, we 
may and do assume y = F(y) = 0, changing coordinates by afhne transformations 
in A. Put i) = t DF (r]') and write rj' and f] as 



Recall that F is written in the form (8) in a neighborhood of and the function / 
in it satisfies (10). In particular, we have fj-f = t (DF^)or]i and fjo = t]q. From the 
assumption (j/f,£') € supp^ m and (F^(y^),n r ) g supp\I/ m ', we have 

( 34 ) WW 2 e supp ^ m ' WW 2 e supp ' 

To proceed, we consider the position of the point ^/(fj) 1 / 2 in E 2d . We consider 
the cases ml < and m' > separately. In the case m' < 0, we claim 



|f?||)V2 



e I lk|| < a- 1 / 2 • 2 m ' +1 } u c;(i/io). 



To prove this claim, we suppose rjj <G R \ C+(l/10) and show 

ll^tll ^ x-l/2 9 m'+l 
(ll^ll) 1 / 2 - • 

By A-hyperbolicity of F, we have ||?)||| < A _1 ||77f||. Since f) = vb> we a l so have 

INI <».<!. 



ll'Tfll " (H 7 ? 
Combining these two inequalities, we get 

JM_ = iiftHi/a . < A-viinjiiv • ll?? t ||1/2 < A -v 2 . 

(||f?||)V2 l^tll (^11)1/2- \W\ (||^||)l/2- (llry'll)^- 

This together with the latter condition in (34) implies the required estimate. In 
the case m! > 0, a similar argument yields 



e I II- > A 1/2 -2 m '- 1 }nc;(i/io). 



(||r?||)V2 

Compare the claims on the position of fj^ / (77) with the first condition in (34) and 
recall the assumption (30). Then we see that there exists a small constant c > 
and a large constant C > 0, which may depend on F, such that, if ra and m' satisfy 
(30) and \m'\ > C, we have 



(lien) 1 / 2 ^'(ii^ii) 1 / 2 



> c • 2 max {l m l'l m 'l> for any 1/2 < /j < 2. 
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If 1/2 < (||£'|l)/(ll»7ll) ^ 2 > the last estimate implies 

W fi\\ > U\ fh\\ > (c/2) • 2 ma ^l m l^'l> • max{(||C'||) 1/2 , (Nl) 1 / 2 } 

and hence the inequality (33) holds with a possibly different constant c > 0. (Note 
that the ratio between ||?7|| and \\r]'\\ is bounded by a constant that depends on F.) 

In the remaining case where either (||£'||)/(||??||) < 1/2 or (||£'ll)/(ll^ll) > 2 holds, 
we can prove the inequality (33) by a crude estimate as follows. Clearly we have 

11^ - > Kll^ll) - <ll^ll>l > | ■ max{<||^||>, <||i?||>} 
in this case. Since (34) implies 

(WW) 1 / 2 > l^tl > 2^- 1 (Wn'W) 1 / 2 > > 2I 1 "'!- 1 

we have also 

max{(||£'||) 1/2 , (||?7||) 1/2 } > i • 2 max ^ ro l'l m 'l>. 

Therefore the inequality (33) holds for a sufficiently small constant c. □ 

Corollary 7.4. There exist a small constant c > and a large constant C > 0, 
which depend on F, such that, if m and m' satisfy (30) and \m'\ > C, we have 

K{x h ^z hmy )>c-2^ m ^ m '^ 

for any (x t ,£) G supply {z^rf) G supp(* m / • X hyp ) and y G suppg. 

Proof. From the assumptions (zf,rj) G supp^m' and (xf,£) G supp* m and from 
invariance of ^ m with respect to the natural action of the transformation group A, 
we have 

(yh tDA (o,x f - yi )(0) = (j/t>£ - £o(ao(a;t) -ao(yt))) e su PP*m and 
(-Ft^t)-*- ^,^-^^))^)) = (-^(yt)' 77- %(«o(^t) - ao(-Ft(2/t)))) e supp* m -. 
Hence we can apply Lemma 7.3 to the setting 

£' = £ - £o(£*o(a;t) - ao(yt))' v' = V~ Vo(uo{zf) - a (-FtO/t)))' 

and obtain the estimate (33) as the conclusion. Note that the estimate (33) implies 
in particular that there exits a constant d > 0, which depends on F, such that 

(35) max{(||e'||), (h'H)} > c' • 2 max <l™l,l™'l} • max{(||C'||) 1/2 , < ||V II > 1/2 > ■ 
Since we have 

||£-£'|| = |£o| • ||ao(a;t) -ao(j/t)ll = l&| ■ Ikt -J/tll 

and 

lh - VII = hoi • ||ao(^t(yt)) - «o(^t)ll = M • ll-Ft(yt) - bli- 
the estimate (33) implies also that we can take a small constant c" > 0, which 
depend on F, so that either of the following three inequalities holds: 

(a) 1^ - C\\ = |£o| • lk t - 2/tll > C" • 2 ma *{l m l<l m 'l} • max{(||e'||) 1/2 , < 1 1 V 1 1 > 1/2 } , 

(b) \\V-Vl = M •||F t (y t )-z t || > c " . 2 ma ^l m l I > - ma X { 1 1 > 1/2 , (l^'ll) 172 }, 

(c) fDF v {r]) - £|| > c" • 2 max {l m l-l m 'l> • max{(||£'|| 1/2 ), (h'|| 1/2 )}. 
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Further, from (35), we may take the constant c" > above so small that, if neither 
of the inequality (a) nor (b) above holds, we have 

m^{\\eUvV>l^{(U\\),(\\v\\)}. 
Hence, with such choice of the constant c" > 0, we always have either (a), (b) or 

(d) W'DFoir,) - £|| > (c"/2) ■ 2 ""«{M.KI} . ma x{(||C||) 1/2 , (N|) 1/2 }. 
Clearly the inequalities (a), (b) and (d) imply respectively that the second, third 
and fourth term in the definition (27) of «(•) is so large that the conclusion of the 
corollary holds for a sufficiently small constant c > 0. □ 

7.4. Proof of Lemma 7.2. From Lemma 6.1 and Corollary 7.4, for arbitrarily 
large \x > 0, there exists a constant > 0, which may depend on F and g but not 
on m nor to', such that 

|tW(*t.OI < C M -2-"- m »{M.KI> f K^x^z^u^z^dz^ 

Jsupp* m , nsupp x hyp 

where 

Note also that the support of v m>m > is contained in supp\Jj m . Hence, by the Schur 
test, the conclusion of Lemma 7.2 follows if we show 

Claim. There exists a constant v > 0, which depends only on d and r, such that 
Jr„(a; t ,6z t ,»7)da; t de < C ■ 2"- nu «{M.Kl} 



/ 

J su 



for (z-\,n) E supp^TO/ n suppX nyp and 

K^x^^z^^dz^dr] < C- 2^ max {l m M" l 'l} 

/supp* TO / n supp x hyp 

for (x^,£) e supp^ m; where C > is a constant that does not depend on to nor 
to'. 



For (x|,£) e supp^m, we have that 

||£t-£o-ao(a>t)|| < |||t ~ Co ■ «o(^t)l < 2 m+1 



(2 • max{|Co|, \\^ £o ■ a (z t )ll}> 1/2 " <(£ 2 + «o(^ t ) II 2 ) 1/2 > 1/2 

and hence in particular that 

liet-eo-ao(^t)ll<2 2|m|+4 -(eo) 1/2 . 

Similarly, for {z^rj) e supp* m , we have 

||r7+ -T7o -« (^+)|| < 2 2 l™'l+ 4 • ( % ) 1/2 . 
On the other hand, if (z^,rj) € supp Jr nyp , we have 

(VoV < \\V1 - Vo ■ a (z)\\ 
by definition. Hence, for (z-\,rj) € supp^,^ n suppJT nyp , we have 

\no\ < 2 (i m 'i+ 4 )/(--( 1 /2)). 

We can show the claim above just by calculating the integrals in the statement 
using these estimates. 
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8. The central part 

In this section, we deal with the central part of the transfer operator. As in the 
last section, we consider the situation assumed in Theorem 3.3 and use the notation 
prepared in the previous sections. We will prove 

Proposition 8.1. The central part £ c t r of the transfer operator £ extends naturally 
to a bounded linear operator £ ctr : -ffaniso ~* ^Iniso- Further, there exists a constant 
Co > 7 which does not depend on F and g, such that, if we take sufficiently large 
number for the constant N according to F and g, the operator norm of the extension 
£ctr : #I„ iso -> #I„ iso is bounded by 

(36) C • max{A(F, 5 ), WgW^ ■ X~ r ■ A(F,g)}. 

Clearly this proposition, together with Lemma 6.3 and Proposition 7.1, com- 
pletes the proof of Theorem 3.3. 

8.1. Beginning of the proof of Proposition 8.1. Now we begin the proof of 
Proposition 8.1. For the proof, it is enough to show that the central part £ ctr of 
the lift C extends naturally to a bounded operator 

£ ctr : L 2 (R 2d ffi R 2d+1 ; W a r niso ) L 2 (R 2d ffi R 2d+1 ; W a r niso ) 

and that the operator norm of the extension is bounded by (36). In other words, 
it is enough to prove 

(37) ||L|| i2 < C ■ max{A(F,. 9 ), • A~ r • A(F,g)} 
for the operator 

L : S(R 2d 8 R 2d+1 ) -+ S(R 2d 8 R 2d+1 ), hu = W r aniso ■ Z ((W^J- 1 • A ctr , • u) 
where (and henceforth) we set 

Actr.o^f 7 ?) = A ctr (z t ,?7) • (1 - X Q (z h n)) 
for simplicity. We write the operator L as an integral operator 

(38) hu{xt,Z)= J WLiSoi ^ ^gf^ ~ • *(*t. 6 *t, vHz h V )dz,d V 

where K(xj,£; z^v) i s the kernel of C given in (25). Note that, if we perform the 
integration with respect to the first variable yo in y — (yo,J/t) m (25), we obtain 
the following expression of the kernel K(xf, £; Zf, rf): 

(39) K(x h £;z hV ) = j g(Zo -%,l/ t ) ■ *(*t> &,*t>W J/ t ) ■ e^t^t^t)^ 
•e 



where 



and 



$fx + fn-z, rm-vt) = ' . e -(Co)kt-ml 2 /2-( n o)|f t ( yt )- zt | 2 /2 

^W'SO.Zt'W'W 7T d -(27r) 2d 

T(s t ,f t ; * t , r? t ; 2/ + ) = Ct ' ((^t/ 2 ) - 1/t) + ft ' (^tfet) - (V 2 )) + % • /(»t)- 
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8.2. Almost orthogonal decomposition of the operator L. In this subsection, 
we decompose the operator L into countably many operators, which are "almost 
orthogonal" to each other and then reduce the claim (37) to a similar claim for each 
of them. 

First we consider a simple partition of unity on the real line 

{q k (t) :l4[0, l]} fceZ) q k (t) = X (t - k + 1) - *(t - k + 2). 
Then, pulling back this partition of unity by the homeomorphism 

we define another partition of unity 

{q k [0,l]} fcez , 
The support of q k for k > (resp. k < 0) is contained in the interval 

p - (2/3)) 2 , (k + (2/3)) 2 ] ircsp. [—{k + (2/3)) 2 , -(k - (2/3)) 2 ]). 
Hence we can take a small constant c > such that 

(40) d(supp q k , supp q~h' ) > c- max{fc, fc'} whenever \k — k'\ > 2. 
We decompose the operator L into countably many operators 

IU : S(R 2d 8 R 2d+1 ) -> S(K 2d © K 2d+1 ) for fe e Z 

defined by 

Lfeit = h(q k ■ u) 

where we identify q~k with the function (arf,£) M> g fe (£ ) on R 2d ©R 2d+1 . Note that 
the operator with fe 2 < N/2 vanishes because so does the term X ctY .o(z, rf'QkiVo) 
in its definition. 

The operators Lfc are almost orthogonal to each other in the following sense. 

Lemma 8.2. For arbitrarily large v > 0, there exists a constant C v > 0, which 
may depend on F and g, such that, if \k — k'\ > 2, we have 

|(L fc u,L fe ^) L2 | < C v ■ m&x{k,k'}~ v • ||u|| L 2 • \\v\\ L 2. 

Proof. We estimate the kernel of L£ o Lj./ by using Lemma 6.1 and (40), and then 
apply the Schur test to obtain the conclusion. We omit the details as it is straight- 
forward and tedious. □ 

We next show that the required estimate (37) follows if we show 

(41) ||L fe || i2 < Co ■ max{A(F,. 9 ), Wg^ ■ A~ r • A(F,g)} 

for all k. Take a function u <E S(R 2d M 2d+1 ) arbitrarily and set u k = l S uppg fe ■ u. 
Since the intersection multiplicity of supp^fc is bounded by 2, we have 

(42) £lNI| 2 <2|M|£ 2 . 

k 



Vt, if t > 0; 

-V^i, ift<0, 



q~k{t) = qk° l{t)- 
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Let us write ||Lu||£ 2 &s 

k,k' 

< ^2 ||LfeUfc|| £ 2 • \\h k ,u k/ \\ L 2 + ^2 (h k u k ,L k >u k ') L 2. 

\k-k'\<l \k-k'\>2 

From Lemma 8.2, (42) and the fact that h k with k 2 < N/2 vanishes, the second sum 
on the last line above should be much smaller than ||u||^2 in ratio, provided that 
the constant N is sufficiently large. For the first sum, the estimate (41), together 
with (42), will yield the estimate 

]T ||L feUfe || i2 ||L fe , Ufe ,|| L2 < £ l|LfcMfclli2+ 2 " Lfc ' Mfc,|li2 <3^||L fcMfc ||i 2 

|fc-fc'|<l |fe-fe'|<l k 

<3(C max{A(F, 5 ), y^-* A(F, g)}) 2 £ ||u fc ||£ a 

k 

< 6 (C max{A(F,. 9 ), A(F, g)}) 2 \\u\\\ 2 . 

Therefore the required estimate (37) follows from (41). 

Next we will go through a similar procedure as above, but this time we consider 
a partition of unity in the space variables. Below we take and fix an arbitrary fceZ 
such that k 2 > N/2. Take a partition of unity on R 2d , 

Ofe,k : R 2d -> [0, 1], k = {k u k 2 , • • • , k 2d ) € Z 2d 

defined by 

2d 

Qk,k( x i) = Yilkjik 1 ^ 6 -Xj) for .x t = (x 1: x 2 ,--- ,X2d) 

3 = 1 

where q k (-) is the function introduced in the beginning of this subsection and S > 
is a small number that will be specified later. Note that the supports of Q k ^ and 
QkM' intersects only if maxj \ki — k[\ < 1, and otherwise we have 

<i(supp(2fe,k , supp(3fe, k ' ) > c • \k\~ 1+s ■ max - A;-| 

l<«<2d 

for some small constant c > independent of k and k. 

By using the partition of unity {Qfc,k}kez 2d 7 we decompose the operator into 
countably many operators 

Lfe,k« = Lfc(Qfe,k • u) = h(q k ■ Q fe , k • u), for k e I? d , 

where we identify Q k M with the function (x,£) ^ Q k ,k(x) on M. 2d ©R 2d+1 . 
Arguing in the similar way as in the proof of Lemma 8.2, we can show 

Lemma 8.3. For any v > 0, there exists a constant C v > 0, which depends on F 
and g, such that, if 

d(k,k') := max \k t - k'A > 2, 

it holds 

|(L fc ,k«,L fcik /«) L 2| < C v ■ \k\~ 5v • rf(k,k')- v • ||u|| La • ||«|| L2 . 
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Then, proceeding similarly to the argument in the paragraph succeeding to 
Lemma 8.2, we see that the claim (41) follows if we prove the estimate 

(43) ||L feik || < C -max{A(F,. 9 ),||. 9 || oo -A- r -A( J F 1 , 3 )} for any k e Z and k e Z 2d . 

In conclusion, we reduced the required estimate (37) on L to the uniform estimate 
(43) on the operators L^k- 

8.3. Approximation by linearization. In this subsection, we prove the estimate 
(43) and finish the proof of Proposition 8.1. By changing the coordinates by ele- 
ments of the transformation group A, we may and will assume that 

k = and F(0) = 



without loss of generality. Let B : R 2d — » Br be the linearization of Fj at the 
origin 0, that is, we set B = (DF^) . Note that it satisfies the conditions (B1)-(B3) 
in Subsection 5.3. 

The operator L^ is an integral operator 

(44) UM*hO = J ]%r"(^j • K k (x^; z hV ) u(z hV ) dz^dr, 
where 

(45) K k (x h ^z h rj) = q k (r]o) ' Qk,o{ z l) ' X ctlfi (z^,r]) ' K ( x h& z h v)- 

and K(xj,£;Zj,r}) is the kernel of £ given in (39) (or (25)). 

As an approximation of the operator L^o , we introduce another operator U k 
that is defined by (44) and (45) but with 

' vw" ln 44 replaced by ^ T ), 2 and, 

• K(x^,^;z\,ri) in (45) replaced by 

K'(x h ^z hV ) = [ g^ - m ,0)-^ k (z r ,x r ,y^-e iT '^^^d yi 



where <fr' k and r' k are defined respectively by 
and 

^(x t ,e t ;^t^t;2/t) = £t • ((*t/ 2 ) - vt) + m ■ ( B (yt) - (V 2 ))- 

Compare the definition of the function K'(-) above and that of K(-) in (39). To 
get the operator h' k from h k ,o, we replaced 

• the diffcomorphism F by its linearization at the origin 0, 

• the function £(£ ,2/t) b y £(£o,0), 

• £ and T] by k 2 , 

and ignored the function /. 

Below we first show that the operator h' k satisfies the estimate (43) and then 
show that Lfc i0 is well approximated by . 
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Lemma 8.4. There exists a constant Co > 0, which does not depend on F nor g, 
such that 

\\K,o\\ < Co ■ max{A(F,. 9 ), yU ■ \~ r ■ A(F,g)} for any k e Z. 

Proof. In the proof below, we ignore the term q k (r]o) • Qk,o(z,r]) ■ X ctI fi(z,r]) in 

(45) , because the multiplication by such a function does not increase the L 2 norm 
of functions. Let us introduce three operators: 

G : L 2 (R) -> L 2 (M), Gu{t) = J g(t - t', 0) • u{t')dt' , 

S k : L 2 (R 2d ffiR 2d ) -> L 2 (M 2d ®R 2d ), S fc «(x t ,^ t ) = u^- 1 ^,*;^) 
and 

W : 5(M 2d M 2d ) -+ S(R 2d ffi R 2d ), WuCr t , f t ) = W r miso (x h (fc 2 , £ t )) ■ «(ar t ,€t)- 

We identify L 2 (R 2d R 2d+1 ) with the tensor product L 2 (R) <g> L 2 (R 2d ffi R 2d ) by 
the natural extension of the correspondence 

L 2 (R) ® L 2 (M 2d M 2d ) 9 u ® w <— > ¥>(s t , : = "(Co) ■ v(*t> ft) e l2 ( mM © rM+1 ) 
where £ = (£oift) - Then the operator h' k is identified with the tensor product 

G®{W olSkY 1 oC B o SkoW- 1 ) : L 2 {R)®L 2 {R 2d ®R 2d ) -> L 2 (M)®L 2 (R 2d ®R 2d ). 

Since the operator G is just the multiplication by 5 viewed through the inverse 
Fourier transform, we have 

\\G:L 2 (R)^L 2 (R)\\ = sup \g(y ,0)\. 

vo eR 

Since S k is a unitary operator, the operator norm of the operator 

W o S- 1 o L B o S* fc o VK- 1 : L 2 (M 2d © M 2d ) -> L 2 (R 2d ® M 2d ) 
should be same as that of 

(46) Z B : L 2 {R 2d © M 2d ; W fe2 ) -> L 2 (R 2d ® R 2d ; W k2 ) 

where W s : R 2d © R 2d — > R is the function defined in (19). As we discussed in 
Subsection 5.3, the operator norm of (46) equals that of 

L ®L : L 2 (R 2d ;V k 2)<E)L 2 (R 2d ) -> L 2 (R 2d ; V fe2 ) ® L 2 (R 2d ) 

and, from Lemma 5.5, is bounded by C ■ maxj^B)- 1 / 2 , d(B) 1 / 2 ■ X~ r }. 

Therefore, noting that d(B) is proportional to det(Di<n|.E+), we conclude that 
the operator norm of h' k with respect to the L 2 norm is bounded by 

Co - ( sup \g(yo,0)\) ■ m & x{d(B)-^ 2 , d(B) 1 ^ 2 ■ \- r } 
\yoeR / 

< C max{A(F, 3 ), \\g\U ■ X~ r ■ A(F,g)}. 
This completes the proof of Lemma 8.4. □ 

The last step of our proof is the following approximation lemma. Recall that our 
construction depend on the constant N > and that both of L/^o and U k vanish 
if k 2 < N/2. 
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Lemma 8.5. For any e > 0, we may take the constant N so large that 
||L fei0 -L' M || < e forallkeZ. 

Proof. We write K{x^, £; z^, rj) and JC'(x^, £; z^, rj) for the kernels of the operators 
L/5 j0 and L' fc respectively and estimate the difference between them. We will 
suppose that the point (zf , rf) satisfies 

(47) ||z t || <2V2d-|fc|- 1+5 , \\vo-k 2 \\ <\k\ 1+s , \\ V -r l0 -a (z^\\<\kr+ 5 

because both of )C(xf z^,n) and K,'{x],^; z^,n) should vanish otherwise. (Recall 
that both of the kernels contain the term qk(vo) • Qk,o(zf) • X c t T fl(zj, n).) Also, we 
may and do suppose that the point (x-f,£) satisfies 

(48) ||x t || < \k\~ 1+25 , Uo-k 2 \\<\k\ 1+2S 7 M-Zo-M^)\\<\k\ 2T+25 . 

This is because, from the estimate in Lemma 6.1, the contributions of the parts 
of the kernels on outside of such region to the operator norms of Lfc ;0 and 
are of order 0(k~°°) and hence we may assume it arbitrarily small by taking large 
constant N . For the same reason, we may and do suppose that the integrations 
with respect to the variable in the definitions of K(x^,£; Zf, n) and K'(x-\,^\ z^rf) 
are restricted to the region 

(49) N-y t ll <\k\- 1+S , \\F t (y^-z t \\<\k\- 1+s . 

If we take sufficiently small constant S > according to the choice of r and r, 
one can check that all of the following quantities are bounded by |fc|~ 2 / 3 , provided 
that (47), (48) and (49) hold and that \k\ is sufficiently large: 

\m-Mv\)-m- B (vi)l l£o-fc 2 lM l%-fc 2 IA 2 , 

|fc 2 |x t - y f \ 2 /2 - (£o)|* t - ytl 2 /2| , \k 2 \B(y t ) - z t | 2 /2 - (%>|^tO/t) - ^tl 2 /2| , 
lff(£o - %,2/f) - ff(£o - i7o, 0)|/ (Co - my 2 , and 
|WI niso (x t , (fc 2 ,C t )) - WI niso (x t ,0|/WI niso (x t ,C), 

iw: niso (z t , (k 2 ,^)) w: niso (z t ^)i/w: niso (z t ,r/). 

Further, under the same assumptions, we have that 

|<%,/0/t)>l< W- 2/3 

from Lemma 3.4, and also that 

C -l . | fc |-r((2.- 1)+5 ) < W:nisoMk \^)) < C- |fc|M(2r-l)+«) 

and 

c -l . |fcr (( 2T - 1)+25) < W:niso (x t ,(fc 2 ,e t )) < C- |fc|M(2r-l) + 2 5 ) 

for a large constant C > 0. Therefore, comparing £(x-|-,£; 2^,77) and /C'(x^.,^; z^,rj) 
and using the estimates above, we obtain 

\!C(x h Z;z h n)-lC'(x h Z;z f ,n)\ < C ■ \k\-^ 2 ■ (^0 - Vo)' 2 ■ f #' fc (*t;*t;i>t) <fot- 



Recalling the restrictions (47) and (48) on the range of (z,T]) and (x,£) and em- 
ploying the Schur test, we see that this implies 

\\U,o-K,ohi<C-\k\- 1 /' 1 

provided that S > is sufficiently small. Therefore, for any e > 0, we have 

||L fei o - L fe,o|| L 2 < e 
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if |fc| is sufficiently large, and otherwise we may assume that both of l*k,o and U k 
vanish by letting the constant N be large. □ 



9. The lower estimate 
In this section, we compete the proof of the main theorem by proving 

Pcaa {^ ■■ Kmso(M) -+ K niso (M)) > A*. 

For this purpose, it is enough to show that there exist a small constant c > 0, which 
does not depend on t, and an infinite dimensional subspace H(t) C H£ niso (M) for 
sufficiently large t such that 

(50) H^MILiso > c- ( sup l 9 *W | • \\u\\ r aniso for all u E H(t). 

\xem ^det(-DF*|_E^)(x) J 

We can construct such subspace H(t) as follows 5 ^. First take a point x* = x*(t) 
that attains the supremum in (50) and choose a coordinate chart n a ■ U a — > V a 
so that F f (x*) <E U a - Take large integer m > and a sparse increasing sequence 
{ n k}tLi of integers and then define a sequence of functions on K 2d+1 by 

<Pk(y) = c k ■ $x t ,t k (y) ■ X{m\y-x\) 
where & Xf ^ k (-) is the function defined in Subsection 5.1, 

x = (x ,x t ) := K a (F t (x*)), ik=n k - a {x) 

and Ck is a normalization constant such that ||y>fc||z,2 = 1. If we take large m and 
sufficiently sparse sequence {nk}, one can show the following properties: 

• ipk := &k ° k , k > 1, are almost orthogonal to each other in if^ niso (M), 

• £*(<pfc), fc > 1, are also almost orthogonal to each other in _ff ! [ niso (M), and 

• the inequality (50) holds with u = ipk for k > 1. 

In fact, if rife are riy are apart from each other, so are the frequencies of ifk & n d 
ipk' (resp. £*(<p/c) and £*((/?£<)) in the flow direction and, therefore, they are almost 
orthogonal to each other in 7J^ niso (M). To check the third claim, note that the 
function ipk is localized in a small neighborhood of F'(x*) on which we may suppose 
that F* viewed in the local coordinate is almost linear and g l is almost constant. 
(Notice that we take m and {nk} according to t.) If F* were linear and g were 
constant, one could obtain the inequality (50) for u — ifk by a straightforward 
estimate using the argument in Section 4, 5 and Subsection 5.3. To conclude, we 
employ an approximation argument similar to (but much simpler than) that in 
the proof of Lemma 8.5. (We ask the readers to work a bit to check the details.) 
The infinite dimensional subspace H{t) spanned by {fk}tLi satisfies the required 
property. 



5 )The following argument may be a bit rough but should be easy to put into a rigorous 
argument once we went through the previous sections. 
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Appendix A. Proof of Lemma 5.3 and Corollary 5.4 

We consider the composition J o : L 2 (R 2d ) -> L 2 (R 2d © R 2d+1 ). The 

Schwartz kernel of this operator is 

Calculating a Gaussian integral, we find 

K{x h & v) = (Zo)- d/2 ■ v- d ' 2 ■ e-^o)- 1 ^-^ 2 ^^-^/^ . 5(to - no) 
For given function u e <S(R 2d+1 ), we set u = and u = 2Tu. Then it holds 

(IMIirr) 2 = \\w r - STog-^ufv 
where w r (x,(;) = (||£||) r , and the right hand side can be written as 



\\w r -&o &-H\\ L , = I u(r,) ■ u( v >) ■ K(x h £; rj) ■ K(x h £; rf) ■ (U\\) 2r dx^drjdr,' 
By calculation, we see that 

J K( X1 ,Z; V ). K(x h £; rf) ■ {£) 2r dx^ = S( V rf) ■ j {i) 2r ■ & 



and that, for some constant C > 0, 

e -<»7o} _1 l»7t _ £tl 



Therefore we have 

(IMI^) 2 = \\w r ■ sr o ^-^m, < c|K • uf L , = c\\uf Hr 

Next we show the estimate in the opposite direction. Note that we may write 

\\u\\ 2 H r = \\w r ■ & o £?*u\\ 2 L2 as 

\\w r -^°.?*u\\ L 2 = J u(x, • u(x> v £>) ■ K(x h £; r,) ■ K(x' v ?; rj) ■ {\\r,\\) 2r dx^drjdr,' 
Since 

J K(x h ^ V ) ■ K(x' p C;v) ■ (H\) 2r dx^ 

= 5 -^0- J , (& ^ ■ (\\v\\r ■ e-M-^-w'-M-^-Wd,, 

and since 

7^7^ J s(Zo m) ■ (h\\) 2r ■ e-^^-^-^-^-^dn < c(U\\y 

for some constant C > 0, it holds 

\\ u \\ 2 Hr = |K- j^o <r*\\i a < c\\w r ■ u\\ 2 L2 = c{\\u\\' Hr f. 

This finishes the proof of Lemma 5.3. 

Corollary 5.4 is essentially a consequence of Lemma 5.3 and the fact that there 
exists a constant C — C(K) > for each compact subset K <e R 2d+1 such that 

C , - 1 <||£||)- 2r < WI niso (x t ,0 < C(U\\) 2r for all x t e K and £ e R 2d+1 . 

Actually, even if the support of u is contained in a compact subset K, the support 
of STu will not be contained in K x R 2d+1 . But a tedious (and rather standard) 
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argument using the fact that 2?u decay rapidly outside the subset K x K 2d+1 give 
the required estimate. 
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